(12) INTERNATIONAL APFUCATION PUBUSHED UNDER THE PATENT COOPERATION TREATY (PCT) 



(19) Worid InteOectua] Prop^ 
Organization 
International Bureau 

(43) International Publication Date 
24 June 2004 (24.06.2004) 




(10) Internationa] Publication Number 

PCT WO 2004/053830 Al 



(51) Intematicmai Patent ClassiilGation^: G09G 5/00 

(21) International Application Number: 

PCTAJS2003/038961 

(22) International Filing Date: 8December2003 (08.12.2003) 

(25) Filing Language: English 

(26) Publication Language: English 
(30) Priority Data: 



60/43 U662 



8 December 2002 (08.12.2002) US 



(71) Applicant {for all designated States except US)i IMMER- 
SION CORPORATION [US/US]; 801 Fox Lane» San 
Jose, CA 95131 (US). 

(72) Inv«itors; and 

(75) Inv«itore/AppIicants (for US only)z GRANT, Danny 
[CA/CA]; 6678 Chateaubriand, Montreal, Quebec H2S 
2N7 (CA). EID, Jeffrey [USAJS]; 105 Shadwell Drive, 
Danville, CA 94506 (US). ENDO,Shc^chi [JPAJS]; 20268 
Northwest Square, Cupertino, CA 95014 (US). 



(74) Agoits: ALEMANNI, John, C et al.; Kilpatrick Stock- 
ton LLP, 1001 West Fouith St., l^inston-Salem, NC 27101 
(US). 

(81) Designated States (national)! AE, AG, AL, AM, AT, AU, 
AZ, BA, BB, BG, BR, BY, BZ. CA, CH, CN, CO, CR, CU, 
CZ. DE, DK, DM, DZ, EC. EE, EG, ES, FI, GB, GD, GE, 
GH, GM, HR, HU. ID, IL, IN, IS, JP. KE, KG, KP, KR, 
KZ, LC LK, LR, LS, LT, LU. LV. MA, MD, MG, MK, 
MN. MW, MX, MZ, NI. NO. NZ. OM, PG, PH, PL, PT, 
RO, RU, SC SD. SE, SG, SK, SU SY, TJ, TM, TN, TR, 
TT, TZ, UA, UG, US, UZ, VC, VN, YU, ZA, ZM, ZW. 

(84) Designated States (regional)x ARIPO patent (BW, GH, 
GM, KE, LS, MW, MZ, SD, SL, SZ, TZ, UG, ZM, ZW), 
Eurasian patent (AM. AZ, BY, KG. KZ, MD. RU. TJ, TM). 
European patent (AT, BE, BG, CH, CY, CZ, DE. DK. EE, 
ES , FI, PR. GB , GR, HU , IE. rr, LU . MC, NL, FT, RO, SE, 
SI, SK. TR), OAPI patent (BF, BJ, CF, CG, CI. CM, GA, 
GN, GQ. GW. ML. MR. NE. SN. TD, TG). 

Published: 

— with international search report 

For two-Utter codes and other abbreviations, refer to the "Guid- 
ance Notes on Codes and Abbreviations" ti^pearing at the begin- 
ning of each regular issue of ifce PCT Gazette. 



S (54) Title: HAPTIC COMMUNICATION DEVICES 



00 

If) 




195 



140- 



Memoiy 



Operating 


■ ► 


System 





Processes 



AH 



7n 



190 



T 

120 



\ 



150 



'Dcanscdver 



160 



Actuator 



— { 



170 



Audio 



7 

130 



180 



User-Interface 



112 



O (57) Abstract: Embodinjents of the invention relate to methods and systenis for providing customized Tiaptic messaging- to users of 
O iMUMfflield coinmnnication devices inavariety of applications. In onecmb^^ 

^ infonnation includes: receiving an input signal associated with a position of a handheld communication device (100); Artenmrang 
O the position of the handheld communication device (100) relative to a predctennined location; and providing a haptic effect assoa- 
S ated%rilh a distance between the position of the handheld communication device (10(J) and the predetennined tocatwn. In another 
^ embodimMit. a handheld communication device (100) can be used as a two-w^ haptic remote oontrol. 



HAPnC COMMUNICATION DEVICES 

INVENTORS 
Danny Grant, Jeffrey Eid, and Shoichi i£ndo 
CROSS-REFERENCE TO RELATED APPLICATIONS 
This plication claims priority to U.S. Provisional Patent ^plication No. 
60/431,062, filed on December 8, 2002, the entire disclosure of which is incorporated herein 
byreference. 

FIELD OF THE INVENTION 

This invention relates generally to haptic-feedback systems. More specifically, 
embodiments of the present invention relate to using customized haptic effects in a variety of 
plications to convey information to users of handheld communication devices. 

BACKGROUND 

As handheld communication devices become part of everyday life, device 
manufactures and service providers strive to enhance the versatiUty and performance of such 
devices. 

Handheld communication devices in the art (e.g., mobile phones, pagers, personal 
digital assistants (PDAs), etc.) typically use Auditory and visual cues to alert a user when 
incoming messages, such as voice calls and emails, are fecdved. Such auditory and visual 
alerts, however, have the disadvantages of being distracting in some situations (e.g., during 
driving), or annoying in others (e.g., during a meeting or a concert). Althou^ vibratory 
alerts are made available in some communication devices such as cellular phones, such 
vibratory effects cannot be customized or personalized according to appUcations. thus 
conveying Uttle information to the user. A need, therefore, exists in the art for a new sensory 
modality that dehvers information to users of handheld communication devices in a 
personalized fashioiL 

SUMMARY 

Embodiments of the invention relate to methods and systems for providing 
customized "haptic messaging** to users of handheld communication devices in a variety of 
^jpUcations. 

In one embodiment, a method of using haptic effects to relate location information 
includes: receiving an iiqjut signal associated with a position of a handheld communication 
device; detemiining the position of flie handheld communication device relative to a 
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predetennined location; and providing a haptic effect associated with a distance between the 
position of the handheld communication device and the predeteraiined location. 

Ill another embodiment, a method of using a handheld commxmication device as a 
two-way hsptic remote control includes: sending a command signal to a remote device; 
deteraiining if a feedback signal is received from the remote device; and providing a haptic 
effect associated with the determination. 

Further details and advantages of embodiments of the invraition are set forth below. 
BMEF DESOEttPTION OE THE FIGURES 

These and other features, aspects, and advantages of the present invention are better 
understood when the following Detailed Description is read with reference to the 
accompanying drawings, wherein: 

FIG. 1 d^icts a block diagram of a haptic handheld communication device according 
to an embodiment of the present invention; 

FIG- 2 shows a flowchart depicting a meihod of using customized haptic effects to 
convey infomiation to users of handheld comnRinication devices, according to an 
embodiment of the invention; 

FIG. 3 shows a flowchart depicting a method of using h^tic logos to relate 
infomiation to users of handheld communication devices, according to an embodiment of the 
inventioi^ 

FIG. 4 shows a flowchart depicting a method of haptically encoding communication 
signals, according to an embodiment of the invention; 

FIG. 5 shows a flowchart depicting a method of providing h^tic messaging to users 
of handheld communication devices, according to a further embodiment of flie invention; 

FIG. 6 shows a flowchart illustrating a mefliod of providing an interactive virtual 
touch in one embodiment of the present invention; 

FIG. 7 depicts a flowchart illustrating a^iiiethod of carrying out a chat session using 
handheld communication devices, according to an embodiment of the invention; 

FIG. 8 shows a flowchart depicting a method of using haptic effects to relate 
navigation information, according to an embodiment of the invention; and 

FIG. 9 shows a flowchart illustrating a method for providing haptic effects to a remote 
control in one embodiment of the present invention. 



DETAILED DESCRIFnON 
Embodiments described in the foUowing description are provided by way of example 
to iUustrate soine graieral principles of the invention, and should not be construed as limiting 
the scope of the invention in any manner. Ome skiUed in the art would also recognize that 
5 various changes and modifications can be made herein, without dq)arting &om the principle 
and scope of the invention. 

FIG. 1 d^icts a block diagram of a handheld communication device 100 according to 
an embodiment of the invention. It will be ^>preciated that various elements are shown in 
schematic form for iUustrative purposes and are not drawn to scale. It will also be 
10 ^reciated that many alternative ways ofpracticing the present invention exit. AccOTdingly, 
various changes and modifications may be made herein, without departing from the 
principles and scope of the invention. 

Device 100 includes a device body including a housing 110 and a user-interface 112; 
a processor 120; at least one actuator 130 in communication with processor 120; and a 
15 memory 140 in communication with processor 120. Device 100 also includes an antenna 150 
and a transceiver 160, in communication with processor 120. Device 100 additionally 
includes a display module 170 and an audio module 180, in communication with processor 
120. Display module 170 may include, for example, a Uquid crystal device. Audio means 
180 may include, for example, a speaker, a microphone, and the like. 
20 For purpose of illustration in the embodiment of FIG. 1, processor 120, actuator 130, 

and memory 140 are shown to be enclosed within and coupled to the device body. Such an 
illustration, however, should not be construed as limiting the scope of the invention in any 
manner. In alternative embodiments, actuator 130 may, for example, be coupled to the 
outside of housing 110, or embedded in housing 110 via a suitable mechanism. Further, user- 
25 interfece 112 may include one or more user-interface members. As used herein, a user- 
intrafece member includes, without limitation,- a key pad having one or more keys, one or 
more buttons, a touch screen or touch pad, a scroll wheel, a direction pad, a trackball, a knob, 
a miniature joystick, or other user-interface means known in the art. 

Device 100 further includes an API (Application Program Interface) 190, working in 
30 conjunction with an operating system 195. A device driver (not shown) may optionally 
provide an interface between operatmg system 195 and processor 120. 

Memory 140 of device 100 stores a program code that includes instructions to cause 
processor 120 to perforin various tasks. The following description provides some examples. 
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FIG. 2 shows a flowchart 200 depicting a method of using customized haptic effects 
to convey infonnation to users of handheld communication devices, according to an 
embodiment of the invention. At step 210, an input signal associated with an event is 
received. At step 220, a source of the event is detennined and a control signal is selected 
5 based on the determination. At step 230, a control signal is output to an actuator coupled to a 
handheld comihunication device (see FIG. 1 for an embodiment of such device). The control 
signal is configured to cause the actuator to output a h^tic effect associated with the event, 

Furfheiinore at step 240, a collection of hap^c effects is provided, each haptic effect 
being associated with a control signaL For example, niemory 140 of FIG. 1 can store a 

10 program code that includes instructions to generate the control signals (e.g., each 
characterized by a distinct waveform) for rendering the correspondmg haptic effects. Haptic 
effects (along with associated control signals) may also be downloaded or transmitted fix)m a 
remote source, such as a service provider, a network resource, a Web server, a remote 
handheld communication device or computer. Such downloaded or transmitted haptic effects 

15 can be further edited or modified. At step 250. a mapping between an event of interest and 
one of the stored hq)tic effects is received. By way of example, memory 140 of FIG. 1 may 
also store a program code that enables a user to map an event of interest to one of the haptic 
effects as provided, e.g., via user-interface 112 through API 190, where the event may be 
idraitified by its source. At stqp 260, the one-to-one mappings made between various events 

20 of interest and the corresponding haptic effects are compiled into a h^tic lookup table, which 
can, for example, be stored in memory 140 of FIG. 1. 

In the embodiment of FIG. 2, the term "selecting" includes, without limitation, 
looking up a predetermined mapping between die event of interest and a corresponding haptic 
effect based on the source determination, and selecting/generating a control signal that is 

25 configured to render the desired haptic effect associated with the event (e-g., upon bemg. 
appUed to an actuator). Selection can be made based upon the aforementioned haptic lookup 

table, for exanq)le. 

In one embodiment, the input signal may include a communication signal associated 
with a call event, such as a voice call, an e-mail, or a message in text or multimedia form, 
30 whidi may be received via antemia 150 and transceiver 160 of FIG. 1, for example. The 
"source" of a call event may be related to a characteristic fliat d^tinctty identifies or 
characterizes the call event, such as the caUer's phone number, the sender's e-mail addre^, a 
gR5)hical feature or an icon associated with the incoming message, etc. 
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In another embodiment, flie iiq)ut signal may be associated with a reminder event, 
which may be a self-generated message on the handheld commmiication device serving as a 
reminder for a pre-scheduled activity (e.g., an £?)pointment or a meeting). The source in this 
scenario may be associated with the type of a pre-schedvded activity (e.g., a business meeting 
vs. a restaurant r^ervation), or the time at which the pre-scheduled activity takes place. 

In yet another embodiment, the input signal may include a communication signal 
associated with a status event, for exan:q)le, received via antemia 150 and transceiver 160 of 
FIG. 1. Examples pf a status event include, but are not limited to: an advertisement (e.g., 
sale) event, a one-to-one marketing event, a business-transaction event, a stock-trading event, 
a weather-forecast event, a sports (or game) event, ah entertamment event, and an emergency 
(e.g., 911) event. In this scenario, the source may be associated with a characteristic that 
distinctly identifies the sender and/or the nature of a status event, such as the phone number 
of the handheld user's stock broker, the e-mail address of the user's favorite store, the logo 
associated with the user's favorite TV or radio station, and so on. 
j5 In one embodiment, an event of interest can be accompanied by a distinct haptic 

effect, or overlapping haptic effects, conveying to the user customized information such as 
"who is calling," "what is happening," and so on. The user can also be allowed to i^date the 
haptic lookup table, e.g., to mclude new events, and/or to modify the m^pings between the 
existing events of interest and the corresponding haptic effects. 
20 Moreover, a specific h^tic effect can be assigned to any incoming si^ event whose 

source is unknown, so as to alert the user that the incoming message is fiom an un- 
identifiable or sender. 

As used herem. the term "handheld communication device" mcludes, without 
Umitation, a mobile phone such as a cellular phone or a satellite phone, a personal digital 
25 assistant (PDA), a cordless telephone, a pager, a two-way radio, a handheld or portable 
conq)uter, a game console controUer, a personal gammg device, an MPS player, or other 
personal electronic devices known in the art that are equipped with communication or 

networking capabihties. 

In one embodiihent, the aforementioned haptic effects can be used as haptic ringers 
(e.g., counterparts to auditory ring tones) that are customized or personahzed to convey 
infonnation to the user about various events of interest By way of example, a haptic ringer 
associated with a call fiom a loved one (e.g., th6 user's spouse) may cofmprise low-amplitude 
and high frequency vibrations that impart gentle sensations to flie user. In contrast, a haptic 
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ringer associated with an emergency event (such as a 911-caU) may comprise jolt-like pulses 
fliat impart pounding sensations to the user. 

In contrast with conventional auditory ring tones, the aforementioned haptic effects 
(e.g., h^tic ringers) are more desirable in an environment where extraneous auditory signals 
5 are prohibited (e.g., during a meeting or a concert), and/or where it is difficult to distinguish 

auditory signals (e.g., in a loud environment sudi as an airport). The hsqjtic ringers are also 
more suitable in distracting situations such as driving, so that the user of a handheld 
communication device can keep eyes on the road without having to look at the device. 
Moreover, such h^tic ringers convey customized rndformation to the usct, so that the user is 
10 aware of 'Vho is callmg," "what is h^ening," and so on, as the followmg examples further 
illustrate. 

A handheld communication device such as a mobile phone may be configured to 
allow a user to include haptic information or a haptic code in an outgoing communication 
signal, e.g., carrying a voice call, an e-mail, or a message. The encoding of a communication 

15 signal with haptic information may be based on an estabUshed scheme or protocol, and/or on 
a per-system basis. The haptic code is configured to cause a h^tic effect to be output when 
the communication signal is deUvered to another handheld communication device. In one 
embodiment, businesses and organizations may each be associated with a distinct h^tic logo 
(e.g., a particular vibration pattern) and include their haptic logos m various messages sent to 

20 the handheld communication devices of their customers. Such h^tic logos can serve as 
counterparts to conventional logos known in the art, for example. Various status eveiits 
mentioned above may also be transmitted in this manner. By way of example, a merchant 
may include its haptic logo in various advertisement events and business transaction events to 
be transmitted to the handheld communication devices of its customers. Stock brokers (or 

25 brokerage firms), TV or radio stations, and marketing/advertising agencies may likewise 
include their hq)tic logos in various stock-trading events, weather-forecast events, sports 
events, entertainment events, and one-to-one mariceting eyents to be transmitted to the 
handheld users. 

FIG. 3 shows a flowchart 300 dq)icting a method of iising haptic logos to relate 
30 information to users of handheld communication devices, accordmg to an embodiment of the 
invention. A handheld communication device receives an input signal at step 310, the uq)ut 
signal being associated with a status event The handheld communication device extracts a 
h^tic code ftom the ii^ut signal at step 320, where the haptic code is associated with a 
h^tic logo. At step 330, the handheld communication device provides a h^tic effect 
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assobiated with the hsq)tic logo. Step 330 may include providing a control signal to an 
actuator coupled to the handheld communication device, where the control signal is based at 
least in part on the haptic code and configured to cause the actuator to output the h^tic 
effect 

5 In one embodiment, the extracted h^tic code may be directly ^pHed to the actuator 

for rendering the desired h^tic effect. In another embodiment, the haptic code may be 
configured according to a predetemoined scheme or protocol that includes, for example, a 
table of haptic codes (some of which may be associated with one or more haptic logos) versus 
control signals for rendering the corresponding haptic effects. In this way, a processor in the 
10 handheld communication device can look up the corresponding control signal &om the table 
based on the extracted haptic code, and output the selected control signal to the actuator for 
rendering the desired haptic effect. 

hi the embodiments of FIG. 2 or 3, the handheld communication device (or the h^tic 
code) may be programmed such that the haptic effect is output immediately, or at a 
15 prescribed time after receiving the input signal, as desired in^pHcations. The haptic effects 
can also be triggered by, or synchronized with,"6ther occurrences. 

A handheld communication device may be further configured such that some of its 
user-interface members (such, as those described above) are each associated with a haptic 
code, e.g., accordmg to a predetermined scheme or protocol, to one embodiment, some of 
20 these haptic codes may be associated with h^tic effects that emulate expressions or 
bdlaviors, such as "laugh," "giggle." "hug," "high-five." "heartbeat," "pet purring," etc. This 
allows haptic effects to be transmitted and experienced. e.g.. in an interactive conversation or 
a chat session, by pressmg or manipulating such members. 

By Way of example, suppose that user A (termed "Ahce" herein) is engaged in a chat 
25 session with user B (termed "Bob" herem) via their respective mobile phones. In one 
embodiment, when Bob tells Ahce a joke, AKce can respond by sraidmg a "laugh" sensation 
to Bob, e.g., by pressing a key on her mobile phone that is assigned with a haptic code 
corresponding to a laugh sensation. This causes a signal to be transmitted firom Alice's phone 
to Bob's phone, and a corresponding haptic effect to be output to Bob's phone (and thereby 
30 experienced by Bob), hi alternative embodunents, Ahce can include a haptic code in an 
outgoing message (which may also contain a video unage such as a picture taken by her 
mobile phone, and/or a gra^jhical feature such as an emoticon emulating a smiley face) to be 
faansmitted to Bob, e.g., by pressing the corresponding user-interface member. The haptic 
code causes a haptic effect to be output when the message is dehvered to a remote device 
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such as Bob's mobile phone. In one embodiment, the hsqptic effect may be correlated or 
synchronized with the displaying of a video image contained in tiie message. In anotiier 
embodiment, the generation of the haptic effect based on the haptic code may be carried out 
in amanner similar to that described above with respect to the embodiment of HG. 3. 

FIG. 4 depicts a flowchart 400 illustrating a method of a method of haptically 
encoding communication signals, according to an embodiment of Uie invisntion. At step 410, 
an input signal associated with an actuation of a user-interface member is received. By way 
of example, the input signal may be associated with Alice's pressing or manipulating a 
particular user-interface member. At step 420, a haptic code associated with the actuation is 
determined At step 430, the haptic code is included in an output signal, and the output signal 
is sent to a remote handheld communication device. As described above, the output signal 
may also include a message, a video image, and/or a graphical feature. 

A handheld communication device may also be configured such that a h^tic effect, 
along with a message, is output upon a contact with a user-interface member benag made 
(e.g., by a user or an input device). HG. 5 depicts a flowchart 500 illustrating a method of 
haptic message that can be associated with this situation, according to an embodiment of the 
invention. At step 510 of the flowchart 500, a handheld communication device receives an 
input signal. At stq> i520, the handheld communication device outputs a request for a contact 
with a user-interface member coupled to the handheld communication device. At step 530, 
the handheld communication device provides a control signal associated with the contact to 
an actuator coupled to the handheld communication device. The control signal is configured 
to cause the actuator to ou^ut a h^tic effect associated with the iiq)ut signal. Step 520 may 
include having a visual effect displayed, an auditory effect played, and/or a distinctive haptic 
ringer output, which requests a contact with the user-interface member being made. 

hi one embodiment, the input signal in FIG. 5 may include a haptic code, along with a 
message, a video image, and/or a gr^hical feature, etc. For example, the haptic code may be 
configured to cause a "hug" sensation to be output when the video image contained in the 
input signal is displayed. The input signal may also contain a provision or protocol that 
specifies that the incoming message along with the corresponding h^tic effect is output upon 
a cont^t with a particular user-interface, member (p.g., the #5 key) being made. 
Altenwtively, the handheld communication device may detemaine the user-interface member 
to be contacted, before ou^utting incoming message along with the corresponding haptic 
effect 
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m another embodiment, the mpviX signal of FIG. 5 may be associated with a ''virtual 
touch," e.g.. to mimic a handshake, a "high-five," a pat on the back, a pulse or heartbeat 
sensation a pet puiring sensation, or other touch sensations associated with human (and/or 
human-animal) interactions. Jn one scenario, the input signal at step 510 may include a 
"virtual touch indicator," based on whidx the request for a contact with a particular user- 
int^face member is made. Hie virtual touch indicator may be in the fonn of a haptic code, a 
message or other informative means. The control signal at step 530 may be generated. e.g.. 
based on the virtual touch indicator, a haptic code associated with the user-inteiface member 
at play, or other predetermined scheme. The input signal at step 510 may also include a 
virtual touch indicator along with a virtual touch signal for rendering the desired haptic 
effect In this case, the control signal at step 530 may be based on the virtual touch signal. 

Referring back to the chat session betw^ Ahce and Bob. by way of example at Uie 
end of their chat session, AUce may wish to send Bob a "high-five." She sends to Bob's 
mobile phone a signal including a virtual touch indicator, which in turn prompts a request that 
Bob be in contact with a user-interface member coupled to his phone, such as a direction pad 
(e g by putting his fingers on the individual keys of the direction pad), a key pad, a touch 
scre^atracld,all.ajoystick.orthelike. Tte control signal for rendering a haptic effect that 
emulates a "high-five" may be based on the haptic code associated with the user-interface 
member, transmitted with the input signal from AHce. and/or other predetemuned scheme. 

hiteractive virtual touch tan also be engaged between users of handheld 
communication devices, where the manipulatioii of a user-interface member on one handheld 
communication device is transmitted possibly in substantially real-time to another handheld 
device and experienced by its user, and vice versa. HG. 6 depicts a flowchart 600 illustrating 
a method of providing interactive virtual touch in one embodiment of the present invention, 
hi the embodiment shown, a handheld communication device first receives an input signal 
including a virtual touch indicator at step 610. A distinctive haptic ringer may, for example, 
accompany the arrival of the virtual touch indicator, identifying the sender and the nature of 
the input signal. The handheld commmucation device may then perform any necessary 
initialization to enable the communication at step 620. which may also include requesting a 
contact with a particular user.interface manber coupled to the handheld communication 
device at step 625. lUe handheld communication device subsequently receives a virtual 
touch signal in the communication associated with the desired h^tic effect at step 630. The 
handheld communication device provides the haptic effect at step 640. e.g., by applying «ie 
virtual touch signal to an actuator coupled to the user-interface member. 
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In one embodiment, the virtual touch sigU may be associated with the manipulation 
of a user-interfece member on a remote handheld device and transmitted in substantially real- 
time And the user on the receiving end may respond by acting in a similar fashion, so as to 
emulate an interactive touch. Any schemes for deUvering virtual touch to users of handheld 
communication devices may be used. 

H^tic effects can also be used to enhance and complement the information content 
communicated between handheld communication devices. In one embodiment, a plurality of 
handheld commmiication users may be engaged in a chat session via their handheld 
communication devices. The users may each have a graphical representation or avatar 
displayed on other handheld communication devices. Such avatars can also be haptically 
enabled, for example, whereby their expressions and/or behaviors are acconq>anied and 
enhanced by corresponding haptic effects. HG. 7 is a flowchart 700 depicting a method of 
carrying out a chat session using handheld communication devices, according to an 
embodiment of the invention. In the embodiment shown, a handheld communication device 
receives an input signal associated with a chat message at step 710. The handheld 
communication device displays an avatar associated with the chat message at step 720. The 
avatar may be shown on display 170 of FIG. 1, in one embodiment. At step 730. the 
handheld communication device provides a haptic effect associated with the chat message. 
Step 730 may include outputting a control signal to. an actuator coupled to the handheld 
communication device, where the control signal is configured to cause the actuator to output 
the haptic effect In one embodiment, the h^tic effect may be correlated with an expression 
or behavior of the avatar, such as a laugh or giggle, a cry. a pet purring, or the like. 

Handheld communication devices ire increasingly equipped with navigation 
capability, for example, in communication with the Global Position System (GPS) or other 
navigation systems. Haptic effects can also be used to convey navigation information, such 
as positional and/or directional information, to handheld users. By way of example, FIG. 8 
shows a flowchart 800 depicting a method of haptic navigation, according to an embodiment 
of the present invention. The flowchart 800 discloses receiving an irq)ut signal associated 
with a position of a handheld communication device at step 810; determining the position of 
a handheld communication device relative to a predetermined location at step 820; and 
providing a haptic effect associated with the determination at step 830. Step 830 may include 
outputting a control signal associated with the determination to an actuator coi^led to the 
handheld communication device, the control signal being configured to cause the actuator to 
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output the haptic effect. Further, the ii^ut signal at step 810 may be received from GPS, a 
digital compass, or other navigation systems known in the art 

hi one embodiment, the haptic effect may be associated with a distance between.the 
position of the handheld communication device and a predetermined location (termed 
"destination" herein). For example, the haptic effect may include a vibration having a 
magnitude and a frequency, where at least one of the magnitude and the frequency decreases 
as the distance from the destination diminishes. Additionally, the haplic effect may be 
configured to convey a quantitative measure of the distance. By way of example, the haptic 
effect may include one or more pulse or jolt sensations, where the number of pulses is 
proportional to the number of miles between the position of the handheld device and the 
destination. 

Processors described above (including processor 120 of FIG. 1) can include, for 
example, one or more digital logical processors capable of processing input, execute 
algorithms, and generate output as necessary to perforai various tasks, such as those 
15 described above. Such processors/controllers may include a microprocessor, an Application 
Specific Integrated Circuit (ASIQ, and state machines. Such processors include, or may be 
in communication with, media (including memory 140 of FIG. 1). Such media include, for 
example, computer readable media, which stores program code that, when executed by a 
processor, cause the processor to perform the stq)S described herein. Embodiments of 
computer-readable media include, but are not Umited to. an electronic, optical, magnetic, or 
other storage or transmission device enable of providing a processor, such as the processor 
m a web server, with computer-readable mstructions. Other exampleis of suitable media 
include, but are not Umited to, a floppy disk, CD-ROM, magnetic disk, niemory chip, ROM, 
RAM, ASIC, configured processor, all optical media, all magnetic tape or other magnetic 
25 media, or any other medium from which a computer processor can read. Also, various other 
forms of computer-readable media may transmit or carry instructions to a computer, 
including a router, private or pubUc network, or other transmission device or channel. 

Program code and associated appUcation programs related to various appUcations may 
also reside on a remote source, such as a network resource, a Web server, a remote handheld 
30 communication device or conq>uter, which can be transmitted or downloaded to a handheld 
communication device on a regular or predetermined basis. Haptic effects (along with 
associated control signals) can also be downloaded or transmitted from a remote source, as 
described above. 
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Actuators described above (incltiding actuator 130 showi in FIG. 1) can include, for 
example, a pager motor, an eccentric rotating motor, a haimonid eccentric rotating motor, a 
voice coil, a solenoid, a resistive actuator, a piezoelectric actuator, an electro-active polymer 
actuator, or other types of active/passive actuators suitable for generating haptic effects. U.S. 
Patent Nos. 6.429.846 and 6,424,333 disclose further details relating to some of these 
actuators, both of which are incorporated in fiiU herein by reference. In some embodiments, 
one or more actuators may be implemented in a handheld commimication device, configured 
to deUver appropriate h^tic effects. It will be appreciated that various control schemes can 
be devised accordingly, for controlling the actuator(s) in a manner that best achieves the 

desired haptic effects. 

Referring back to FIG. 1. In one embodiment, actuator 130 inay be coupled to 
housing 110, thereby imparting haptic effects thus generated to fee device body, H^tic 
ringers (or alerts) described above may be delivered in this manna:, for instance. In another ' 
embodiment, actuator 130 may be coupled to user-interface 112 of the device body. For 
instance, an active and/or resistive actuator can be coupled to user-interface 112 to deliver a 
virtual tluch described above. One or more actuators can also be coupled to user-interface 
112. for example, to convey a virtual touch such to a user. In yet another embodiment, a 
plu^ty of actuators can be coupled to housing 110 as weU as user-interface 112. In 
addition, one or more actuators may also be coupled to a headset, a wristband, or other 
accessory means associated with a handheld communication device. 

Embodimentis of the invention include tiie following. 

In one embodiment, an individual (or ''Bob") can have a mobile phone according to 
the invention. The mobile phone also has an e-mail capabiKty. for example, including both 
"receive" and "send"). The mobile phone is corifigured to provide a plnraHty of haptic 
effects, e.g., by including ^propriate hardware (such as actuators described above), and 
program code. Bob can program the mobile phone, for example, via user-interface 112 
through API 150 shown in FIG. 1, by inputting various events of interest and associating each 
with a distmct haptic effect Thus, when an event of interest is subsequentiy received, the 
mobile phone provides flie corresponding haptic effect 

In one embodiment, Bob's phone includes programming that provides a first haptic 
effect when an input signal is received from the mobile phone of Bob's wife (or "Alice"). 
Bob's phone also inchides programming that provides a second haptic effect that is different 
and distinct from the first h^tic effect, whm an input signal is received from the mobile 
phone of Bob's supervisor at work (termed "Carol" herein). Bob's phone is fimher be 
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configured to provide a tbird hq)tic effect thit is different fiom the two mentioned above, 
e.g., when an e-maU is received from the e-mail address of Bob's stock broker (where the e- 
maa contains a "smiley-face" emoticon. for instance). The third haptic effect can be a 
vibration with high magnitude and short duration, e.g., to emulate a -Tiigh-fLve." 

In another embodiment. Bob can be watching a movie in a theater with his mobile 
phone in his pocket It is set to make no noise, because Bob is in a theater. While Bob is 
watching the movie, Bob's mobile phone vibrates wifli the second haptic effect mentioned 
above. Bob chooses to ignore the call, because he does not wish to speak with his supervisor 
at a movie theater. Later. Bob's mobile phone vftrates with the first h^tic effect Bob wants 
to speak with AUce, for example, to make plans to meet later. So Bob Mswers the phone and 

quickly exits the theater to talk with Alice. 

Bob's mobile phone can also include a personal schedule/calendar appUcation. After 
speaking with Alice, Bob can enter an entry in the calendar at the 7:00PM time maik - "M.cet 
AUce". Bob can also choose a fourth haptic effect to associate with the calendar entry. The 
mobile phone can be programmed to output the fourfli h^tic effect fifteen minutes before the 
time entry of the calendar (i.e., at 6:45PM). 

Bob's mobile phone can be equipped with GPS capabiUty, along with an associated 
appUcation program for location deteraiination. Bob can. also store addresses of various 
locations of interest in the appUcation program: Ih one embodiment Bob can be on the road. 
Bob's mobilje phone vibrates with a distinct fiftti h^tic effect Bob recognizes the fifth 
haptic effect being associated with the hq)tic logo of his favorite electronics store. He then 
checks with the appUcation program, and receives a sbrth haptic effect associated with the 
distance between his current position and the store location. Sob then decides to make a stop 
at the store. 

A haptically-enabled handheld communication device of the invention may be further 
used as a two-way haptic remote control, for example, for controUing a remote system such 
as a Television set or a multimedia system, hi one embodiment, the events as referred to 
above may be related to program channels shown on the remote system, each identified by a 
channel number (which may be used as the "source"), for instance. The corresponding haptic 
effects may be customized on a per-channel basis. Such haptic effects can serve to inform a 
user as to whidi channel is on, as a user is channel-surfing by way of this haptic remote 
control, so that the user need not to look the display screen. 

FIG. 9 depicts a flowchart iUustcating a method for providing haptic effects to a 
remote control in one embodiment of the present invention In flie embodiment shown, the 
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remote control sends a conunand signal to a ranote system at step 910. As with a 
conventional remote control, the signal may or may not reach the television. The reniote 
control then determines whether a feedback signal has been received at step 920. If the 
remote control receives a feedback signal, the remote control provides a first haptic effect at 
step 930. If not, the remote control provides a second lis^tic effect at step 940. 

The first haptic effect can be further customized according to the received feedback 
signal. Jn one embodiment, the remote systemi provides infomiation (e.g., via the feedback 
signal) to the remote control regarding the state of the display, e.g., based on a predetermined 
scheme. The remote control may use the information to determine a corresponding haptic 
effect to provide at step 930. hi altemative embodiments, the remote system may determine 
the appropriate haptic effect to provide and include a corresponding haptic code in the 
feedback signal. The remote control provides the haptic effect at step 930 based on this 
haptic code. 

The foregoing description of the preferred embodiments of the invention has been 
presented only for the purpose of illustration and description and is not intended to be 
exhaustive or to limit the invention to the precise forms disclosed. Numerous modifications 
and ad^tations thereof will be apparent to those skilled in the art witiiout departing fi"om the 
spirit and scope of the present invention. 
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That which is claimed is: 

1. A method, coiiq)iising: 

receiving an input signal associated with a position of a handheld communication 

device; 

detennining the position of the handheld communication device relative to a 

predetemsined location; and 

providing a hsp^c effect associated with a distance between the position of the 
handheld communication device and the predetemuned location. 

2. The method of claim 1 wherem the input signal is recraved ftom one of a Global 
Position System and a digital compass. 

3. The method of claim 1 wherein the haptic effect includes a vibration having a 
magnitude and a frequency, at least one of the magnitude and the frequency being associated 
with the distance between the position of the handheld communication device and. the 
predetermined location. 

4. The method of claim 3 wherein the at least one of the magnitude and tlie frequency 
decreases with the distance between the position of the handheld communication device and 
the predetermined location. 

5. The method of claim 1 further comprising outputting a control signal associated with 
the deteraiination to an actuator coupled to the handheld communication device, the control 
signal configured to cause the actuator to output the haptic efifect. 

6. A mettiod, comprising: 

sliding a command signal to a remote device; 

detemiining if a feedback signal is received from the remote device; and 
providing a ha?)tic efifect associated with the determination. 

7. The method of claim 6 ftirther comprising providing a first haptic efiFect i f a feedback 
signal is received from the remote device. 

8. The method of claim 7 further comprising providing a second haptic effect if no 
feedback signal is received from the remote device. 

9. A computer-readable medium on which is encoded program code, comprising: 
program code for receiving a signal associated with a position of a handheld 

conununication device; 

program code for determining the position of the handheld communication device 

relative to a predetermined location; and 
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program code for providing a haptic feedback associated wiih a distance between the 
position of the handheld communication device and the predetermined location. 

10. The computer-readable medium of claim 9 wherein the haptic effect includes a 
vibration having a magnitude and a frequency, at least one of the magnitude and the 
frequency being associated with the distance between the position of the handheld 
cconmunication device and tiie predetermined location. 

11. The computer-readable medium of claim 10 wherein the at least one of the magnitude 
and the frequency decreases wifli the distance between the position of the handheld 
communicatioii device and the predetermined location. 

12. The computer-readable medium of claim 9 farther comprising program code for 
outputting a control signal associated with the detennination to an actuator coupled to the 
handheld communication device, the control signal configured to cause the actuator to output 
the hq>tic effect 

13. A computer-readable medium on which is encoded program code, comprising: 
program code for sending a conunand signal to a remote device; 

program code for detOTiiining if a feedback signal is received from the remote device; 

and 

program code for providing a haptic effect associated with the detemiination. 

14. The computer-readable medium of claim 13 further comprising program code for 
providing a first h^tic effect if a feedbatsk signal is received from the remote device. 

15. The computer-readable medium of claim 14 fiirther comprising program code for 
providing a second haptic effect if no feedback signal is received from the remote device. 

16. A data stream embodied in a carrier signal, carrying instractions to: 

receive a signal associated vdth a position of a handheld communication device; 
deteimine the position of the handheld communication device relative to a 

predetermined location; and 

provide a haptic effect associated with a distance between the position of the handheld 
communication device and the predetermined location. 

17. A data stream embodied in a carrier signal, carrying instractions to: 
send a command signal to a remote device; 

detemiine if a feedback signal is received &om the remote device; and 
provide a haptic effect associated wi& the determination. 
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18. The apparatuis, comprising: 

means for receiving an inpat signal associated with a poation of a handheld 
commvmication device; 

means for determining the position of the handheld commvmication device relative to 

a predeterajined location; and 

means for providing a h^tic effect associated with a distance between the position of 
the handheld communication device and the predetermined location. 

19. The ^aratus of claim 18 wherein means for providing includes ah actuator coupled 
to the handheld communication device, the actuator configured to receive a control signal 
associated- witit the determination and ouQ)Ut the haptic effect 

20. The apparatus of claim 18 wherein the haptic effect includes a vibration having a 
magnitude and a fi-equency. at least one.of the magnitude and the frequency being associated 
with the distance between the position of the handheld communication device and the 
predetermined location. 

21. The apparatus of claim 20 wherein the at least one of the magnitude and the frequency 
decreases with the distance between the position of the handheld communication device and 
the predetermined location. 

22. The apparatus, comprising: 

means for sliding a command signal to a remote device; 

means for determining if a feedback signal is received from the remote device; arid 
means for providing a haptic effect associated with the determination. 

23. The apparatus of claim 22 wherein a first h^tic effect is provided if a feedback signal 
is received from the ranote device. 

24. The apparatus of claim 23 wherein a second haptic effect is provided if no feedback 
signal is received from flie rranote device. 

25. The 25)paratus of claim 22 is included in a handheld communication device. 

26. The apparatus of claim 25 wherein means for providing a h^tic effect mcludes an 
actuator coupled to the handheld communication device, the actuator configured to receive a 
control signal associated with the determination and output the haptic effect. 
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